fix for 61123 read_excel-nrows-param-reads-extra-rows #61127
Closed
+492
−1
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue: GH-61123
When reading Excel files with
pd.read_exceland specifyingnrows=4, the behavior differs depending on whether there’s a blank row between tables. For a file with two tables (each with a header and 3 data rows),nrows=4should yield a DataFrame with one header and 3 data rows (shape(3, n)). However:test1.xlsx(with a blank row), it correctly reads the first table (header + 3 rows).test2.xlsx(no blank row), it incorrectly includes the second table’s header as a data row, resulting in a shape of(4, n).This inconsistency occurs because
read_exceldoesn’t properly respect table boundaries when tables are adjacent, despite thenrowslimit.Fix:
pandas/io/excel/_base.pyand related reader modules (_openpyxl.py,_pyxlsb.py,_xlrd.py) to ensurenrowslimits reading to the specified number of rows, excluding subsequent table headers even when tables are adjacent.test_excel_read_tables_with_and_without_blank_rowinpandas/tests/io/excel/test_readers.pyto verify thatnrows=4consistently returns a DataFrame with shape(3, 2)(header + 3 data rows) for both cases.Changes:
nrowswithout parsing beyond table boundaries.openpyxl,pyxlsb, andxlrdengines.Verification:
test1.xlsx(blank row) andtest2.xlsx(no blank row).(3, 2)and only the first table’s data.Steps to Test:
pytest pandas/tests/io/excel/test_readers.py::TestReaders::test_excel_read_tables_with_and_without_blank_row.df1.shape == (3, 2)anddf2.shape == (3, 2)match the expected output.Related Files:
pandas/io/excel/_base.pypandas/io/excel/_openpyxl.pypandas/io/excel/_pyxlsb.pypandas/io/excel/_xlrd.pypandas/tests/io/excel/test_readers.pyCloses #61123